SELECTIVE ENGAGEMENT OF MOTION 
DETECTION \. * 

Inventors: David L. Marvit et al. 

Date Filed: March 23, 2004 

Attorney's Docket: 073338.01 84 (04-50458 FLA) 

Sheet 1 of 1 1 

1/11 



10- 



1W 



INPUT 




DISPLAY 








PROCESSOR 



16 



MOTION 
DETECTOR 

—? 

22 



20 



•12 



18 



APPLICATIONS 
19 



21 



DATABASES 



COMMUNICATION'S INTERFACE 



FIG. 1 



X-AXIS 
ACCELEROMETER 



Y-AXIS 
ACCELEROMETER 



Z-AXIS 
ACCELEROMETER 

7 

24c 



32- 



PR0CESS0R 



FIG. 2 



26a^ 


CAMERA 


28a -vj 


GYRO 


30a -v. 


^-24a 


















26b^ 


CAMERA 


28b^ 


GYRO 


30b^ 


^-24b 


















26c-^ 


CAMERA 


28c^ 


GYRO 


30c^" 



RANGEFINDER 



RANGEFINDER 



RANGEFINDER 



MOTION DETECTOR 



V 

22 




23a- 
23b- 
23c- 



25a- 
25b- 
25c- 



SELECTIVE ENGAGEMENT DEMOTION 
DETECTION 

Inventors: David L. Marvit et al. 

Date Filed: March 23, 2004 

Attorney's Docket: 073338.01 84 (04-50458 FLA) 

Sheet 2 of 1 1 



X-AXIS ACCELEROMETER 
RAW DATA 



2/11 



Y-AXIS ACCELEROMETER 
RAW DATA 



Z-AXIS ACCELEROMETER 
RAW DATA 



CAMERA RAW DATA 



CAMERA RAW DATA 



CAMERA RAW DATA 



34 



16- 
36- 



32 



PROCESSOR 



TRANSLATION X 



TRANSLATION Y 



TRANSLATION Z 



ROTATION X 



ROTATION Y 



ROTATION Z 



I 



PROCESSOR 

— I — 



DEVICE BEHAVIOR 



GYRO 
RAW DATA 



GYRO 
RAW DATA 



GYRO 
RAW DATA 



RANGEFINDER 
RAW DATA 



RANGEFINDER 
RAW DATA 



RANGEFINDER 
RAW DATA 



FIG. 3 



•27a 
•27b 
•27c 



-29a 
-29b 
-29c 




46 48 



■50 



FIG. 5 



SELECTIVE ENGAGEMENT OF MOTION 
DETECTION • - ; 1 

Inventors: David L. Marvit et al. 

Date Filed: March 23, 2004 

Attorney's Docket: 073338.0184 (04-50458 FLA) 

Sheet 3 of 11 

3/11 



X-ACCELERATION 
DATA 



Y-ACCELERATION 
DATA 







AUGMENT X 



T 

68a 




3 1 i 



70^1 
72- 



PROCESS 



DEVICE BEHAVIOR 



Z-ACCELERATION 
DATA 



68C 



FIG. 6 



1 




AUGMENT Z 



J 



60 



j USER "I 
I PREFERENCES « 

69 



80 

\ 



( START ) 



82a DETECT X-AXIS 
ACCELERATION A 



82b- 



FIG. 7 



DETECT Y-AXIS 
ACCELERATION A 



DETECT Z-AXIS 
ACCELERATION A 




86- 



SET ZERO-POINT 
( END ) 



SELECTIVE ENGAGEMENT OF MOTION 
DETECTION * 

Inventors: David L. Marvit et al. 

Date Filed: March 23, 2004 

Attorney's Docket: 073338.01 84 (04-50458 FLA) 

Sheet 4 of 11 

4/11 



FIG. 8 



92 

± 



94 



96 



98 











= 1=1! ! 


F 


fl 


; D ! 


0 


-fl-l — 


— r| -r- 


-fl- 


D { 




D 


0 


! 0 1 


D 


4]-4 


__^Q. \ 


■-Q- 




= U 





± 



+"1 

B 



0 



QtarT) /t/g. p 



100- 

102- 
104- 
106- 

108- 





MOVE DEVICE TO RIGHT 












DISENGAGE MOTION 
SENSITIVITY 








MOVE DEVICE TO LEFT 




\ 


REENGAGE MOTION 
SENSITIVITY 




V 


MOVE DEVICE TO RIGHT 




^-^NEED FURTHER^^YES 



MOVEMENT? 



110 



NO 



SELECTIVE ENGAGEMENT OF MOTION 
DETECTION ' 

Inventors: David L. Marvit et al. 

Date Filed: March 23, 2004 

Attorney's Docket: 073338.0184 (04-50458 FLA) 

Sheet 5 of 1 1 



5/11 




1<K 




132 





/ 


□ 




V 


/ 



10 




FIG. 10A 



133 



n 



135. 136 




137 

L 



C 3) 



UP 



DOWN LER 



RIGHT 



138 




IN 




139 



OUT 



i*7G. JOB 



142 



140 



144 



146 



A 


B 


C 


D 


E 


F 


G 


H 


I 


J 


K 


L 


M 


N 


0 


P 



! B1 


B2 


B3 


B4 


B5 


B6 


B7 


B8 


B9 



B2a 


B2b 


B2c 


B2d 


B2e 


B2f 


B2g 


B2h 


B2i 



FIG. 11 



147 



A 


B 


C 

< 


D 


E 


F 


G 


> 

H 


I 


J 


K 


L 


M 


N 


0 


P 



SELECTIVE ENGAGEMENT OF MOTION 
DETECTION 

Inventors: David L. Marvit ct al. 

Date Filed : March 23 , 2004 

Attorney's Docket: 073338.01 84 (04-50458 FLA) 

Sheet 6 of 1 1 



150 



148 148 



149 



6/11 



A 

< 


B 

> 


C 


D 


E 


F 


G 


H 


1 


J 


K 


L 


M 


N 


0 


P 



151 



148 



J 



152 



A 


B 


C 


D 


E 


F 


G 


H 


I 


J 

i 


K 


L 


1 

m( 


N 


0 


P 



153 



154 



A 


B 


C 


D 


E 


F 


G 


H 


I 


J 


K 


L 

> 


M 


N 




P 



148 



FIG. 12A 



FIG. 12B I 



A 
B 
C 
D 
E 
F 



161 



B2 

< 


B3 




C3 



159 





160 

S 








162 






































164 

X 





































163 



B5 
1 


B6 
i 


C5\ 


C6 



159 



165 



159 



E3 

« 


E4 
i 




F4 



-158 



SELECTIVE ENGAGEMENT OF MOTION 
DETECTION 

Inventors: David L. Marvit et al. 

Date Filed: March 23, 2004 

Attorney's Docket: 073338.0184 (04-50458 FLA) 

Sheet 7 of 1 1 



7/11 




184^1 




SELECTIVE ENGAGEMENT OF MOTION 
DETECTION 

Inventors: David L. Marvit et al. 

Date Filed: March 23, 2004 

Attorney's Docket: 073338.01 84 (04-50458 FLA) 

Sheet 8 of 1 1 

8/11 



236 



232- 
234- 



RECEIVE RAW MOTION DATA 



230 



PROCESS 



DETERMINE MOTION AND ORIENTATION 



T ROTATING ~| T TRANSLATING ~| T ORIENTED AT "1 
' AROUND Z-AXIS ' • ALONG X-AXIS ' ' a,£w ' 

. v , 

237a 237b 237c 



o o o STILL 



i 



237n 



DETERMINE ENVIRONMENT BASED ON MOTION AND ORIENTATION 



, 1 1 f— — — -L ~ — r , 

I FACEDOWN i i CAMIKir > I I nuTDAiM I 

I ON TABLE « I FALLING « I TRAIN , 

I j 1 J - I L 



I ^ rt ~ i 
I 



HELD IN 



T 

239a 



T 
239b 



"V" 

239c 



[_ HAND 



239n 



MAP ENVIRONMENT TO BEHAVIOR 



I ENGAGE "1 T POWER DOWN CHIPS "! 
' MUTE ' ' TO SURVIVE IMPACT « 

L — V J 

241a 241b 



T^^l 

I INCREASE MOTION | 
' ACTIVATION THRESHOLD " 

*- v — J 

241 n 



242- 



BEHAVE 



240 



FIG. 16 



] 

SELECTIVE ENGAGEMENT OF MOTION 
DETECTION 

Inventors: David L. Marvit et al. | 

Date Filed: March 23, 2004 

Attorney's Docket: 073338.0184 (04-50458 FLA) 

Sheet 9 of 11 

9/11 




FIG. 17 



270 

\ 



N 



RECEIVE RAW 
MOTION DATA 



PROCESS 



MAP MOTION 
TO GESTURE 



"0" 



MAP GESTURE 
TO FUNCTION 



BEHAVE 



280 



GESTURE 
DATABASE 



FUNCTION 
DATABASE 



284 



330 

/ 

RECEIVE USER 
INDICATION 
REGARDING 
GESTURE CREATION 



RECEIVE RAW 
MOTION DATA 



PROCESS 



STORE MOTION 
AS GESTURE 



RECEIVE 
FUNCTION MAPPING 
INFORMATION 



STORE 
FUNCTION MAPPING 
INFORMATION 



-332 



•334 
■336 

-338 

-340 



-342 



FIG. 18 



FIG. 21 



SELECTIVE ENGAGEMENT OF MOTION 
DETECTION '* " * ' 

Inventors: David L. Marvit et al. 

Date Filed: March 23, 2004 | 

Attorney's Docket: 073338.0184 (04-50458 FLA) , 

Sheet 10 of 11 



10/11 




FIG. 19 



APPLICATION 4 



312- 



314- 



RECEIVE RAW 
MOTION DATA 



PROCESS 



310 



316^. 



MAP MOTION 
TO GESTURE 




FIG. 20 




320c 



USER 4 



320d 



SELECTIVE ENGAGEMENT OF MOTION 
DETECTION 

Inventors: David L. Marvit et al. 

Date Filed: March 23, 2004 

Attorney's Docket: 073338.0184 (04-50458 FLA) 

Sheet 1 1 of 1 1 

11/11 



10- 



CD 



V 

\ 
I 

/ 

/ 



350 



10- 



/ 



\ 



^352 



10- 



A 



354 



FIG. 22 



370 
\ 



372-v 



RECEIVE RAW 
MOTION DATA 



374- 



376- 



1 



PROCESS 

~r~ 



MAP MOTION 
TO GESTURE 



384 



382- 



MAP GESTURE 
TO FUNCTION 



378 



USER 
SETTINGS 
DATABASE 



GESTURE 
DATABASE 



FUNCTION 
MAPPING 
DATABASE 



386 



USER 1 
o PRECISION 
o NOISE 

o USER-CREATED 
GESTURES 



USER 2 



•379 



FUNCTION 1 
392a 



FUNCTION 2 

— 7 

392b 



1 



FUNCTION 3 
392c 



ENVIRONMENTAL MODEL 



USER 1 

o USER MAPPING 

o USER-CREATED 
FUNCTIONS (MACROS, 
PHONE NUMBERS,...) 



USER 2 



•380 



-387 

USER 
" IDENTITIES 



"V" 

381 







ENVIRONMENTAL MODEL 






CONTEXT 






APPLICATION IN FOCUS 


388 










DEVICE STATE 







•389 
390 
•391 



FIG. 23 



